Papers with long document classification tasks
Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document classification struggle with token limits and fail to adequately model hierarchical relationships within documents. |
| Approach: | They propose a novel model leveraging a graph-tree structure to capture local and global dependencies. |
| Outcome: | The proposed model captures syntactic relationships and broader document contexts without token limits and can handle arbitrarily long contexts. |
Interpretable Research Replication Prediction via Variational Contextual Consistency Sentence Masking (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for predicting research replication are insufficient especially for long research papers. |
| Approach: | They propose to build an interpretable neural model which can provide sentence-level explanations and apply weakly supervised approach to leverage large corpus of unlabeled datasets. |
| Outcome: | The proposed model can provide sentence-level explanations and leverage large unlabeled datasets to boost interpretability and improve prediction performance. |
Efficient Classification of Long Documents via State-Space Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing transformer-based models can only process long documents with limited computational resources due to their quadratic computation time and space. |
| Approach: | They propose to use state-space models for long document classification tasks instead of using sparse or hierarchical structures to solve this problem. |
| Outcome: | The proposed model performs comparable to self-attention models while being 36% more efficient. |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
Revisiting Transformer-based Models for Long Document Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent literature in text classification is biased towards short text sequences . multi-page multi-paragraph documents cannot be efficiently encoded by vanilla transformers based on short text. |
| Approach: | They compare different Transformer-based Long Document Classification approaches to mitigate the computational overhead of vanilla transformers to encode much longer text. |
| Outcome: | The proposed models can process longer text and provide practical advice for long document classification tasks. |
ChuLo: Chunk-Level Key Information Representation for Long Document Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional approaches to truncate inputs, sparse self-attention, and chunking often lead to information loss and hinder the model’s ability to capture long-range dependencies. |
| Approach: | They propose a novel chunk representation method that uses unsupervised keyphrase extraction to group input tokens to retain core document content while reducing input length. |
| Outcome: | The proposed method minimizes information loss and improves the efficiency of Transformer-based models. |